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ABSTPACT 

The administration of the National Test-Equating 
Study in Reading is described as part of a symposium on the study. 
After contacts with each of the State Departments of Education, 
district and LEA superintendents were asked to consider the 
participation of randomly selected schools within their 
jurisdictions. Upon receipt of approval, school principals were 
invited to participate in the study and were asked to appoint a 
School Coordinator. The Coordinator provided the enrollment and 
school descriptive data required for the assignment of tests and the 
shipment of test materials. Score rosters, showing raw scores, 
publishers* national percentiles and stanines, and summary 
statistics, for both subscores and total reading scores, were 
provided to each participating school. Reports were released to 
schools by the first week in June. (Author/KM) 
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THE INVITATIONAL PHASE 



The invitational phase of the Anchor Test Study required invitation 

t 

latters addressed to rhe responsible individuals at three levels: Chief 

1 I 

^ State School Officer, District (Lo^^ax i^duc^tion A^;ency) Superinten^cat, 



ft 

and Ccb'»A Principal. 



Specifications required that approval be received from a higher admin- 
istrative level prior to ^,he issuance, of an invitation to any potential 

i 

^ participant, i.e., it was necessary to obf'in app.r^val fi/om the state's 
J chief school officer before issuing anv District (LEA) Superintendent 
If inviCiitior. Similarly, it was necessary to receive district superintend". it 
^ approval prior to the issuance of school invitations. 
P Stctte liivltdcions 

I p During the second week of October 1971, fifty-two copies of the Chief 

State r»chuol Officer invitation letters were released. After the text had 
been approveil by USOE, the letter*' were prepared by end final copies 

transmitted to 130E for signature- Tho letu/rs were signed by S.P. Marland, 
^ Jr., U.S. Commissioner of Education, and Floyd T. Christian, Presidr^c, 

Council of Chief State School Officers. ETS began the invitational phase 
^ by releasing this initial mailing to "i^^ chief state school officers in 
the fifty states, the District of Coluiauict, and the National Catholic 



Education Association. 

The Chief State School Officer (CSSO) invitation described the study, 
its background, and the benefits to be gained from its accomplishment. The 



^Prepared for presentation at the American Educational Research Association 
meetings, New Orleans, Louisiana, February 25 - March 1, 1973. 
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CSoO was asked to approve the participation of hiy/b^.r -jtate in the Anchoi: 
Test Study, to prepare a letter of endorsement for district superintendents 
in the state, and to appoint a State Coordinator fzr the ptuject. The 
State Coordinator's function was to 1) assure the existence of selected 
schools, 2) provide the names of district sup^irintendents ii. the selected 
school H.»st.:icts, 3) assist in the resolution of any communication problems 
in the c^al^^ah7^-A"V--p^.^^^^ ^f.|-y^^.^ ETS and the Chief Snare School 

Officer. 

/s endorsements were received from each chief state school officer, it 
was possible Lo coutrct thf person designated as State Coo/dinator* A letter 
acknowledging this individual's appointive was prepared and sent to him/her, 
together with a computer-prepared listing of the selected districts -.n the 
state. State coordinators were asked to supply the names and addresses of 
thf^ a-iistrini superintendents whose schools had bet.n zzlt^ztzl fcr participation 
in either the primary or back-up samples. 

District Invitations 

Upo.» receipt of thr district superintendents' names and addresses, it 
wa3 possiDxfc t-> begin the district invitation process. Since the beginnin- 
or uuxs phase depended on replies from t!:e st£»:e coordinators, the release 
of district invitations varied from state to state, beginning in November 
1971 and lc«sting through January 1972. 

^xctrxc\ invitations consisted of four components: e. letter of invi- 
tation, a list of selected primary and back-up schools within the district, 
four descriptive brochures, and a reply postcard. 
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District superiat€*nde:itd were asked to approve the part i ciyation of 
selected schools in their districts and to appoint a District Coordinator, 
who v/ould verify tlie existence of selected schools (as well as the existence 
of grades 4, 5, and 6), provide the names and addresses of school principals, 
help with the distribution of school invitations, and assist in the resolution 
of any co-amunication problems. Each superintendent was sent a 3i t ot schools 

selected to participate, as well rs c list of those schools selected ohjx 

back-up basis, i.e., schools which would be invited only if a speciii : 
originally selected (primary sample) school was unable to partici.pate. Most 
of the district Invitations vere released during late November and December, 
although zjxrc lew back-up district invitations were released ac late as HaiTch. 
The State Coordinator was iic.ifier. when the invitations for districts in his/ 
her state were released. A return reply control system was maintained so 
that ncnrespondents 2ov''^ r»nn*-ar**ed, and appropriate bach-tzp dictriccs 
invited. 

At weekly intervals during the di^r^rict invitation phase, computer- 
produced status reports were released. Ejch state coordinator received a 
yeport for the districts within his/h-.r state. The report shoved districts 
agreeing f/* parfcipate and the name of the individual appointed as District 
Coordinator. In addition, the report listed those districts invited but not 
yet responding, those declining the invitation, and those containing back-up 
schools which had not yet been i'Avited. 

Although the school, rather than the district was the primary sampling 
unit for the Study, some ov*=rall figures on district participation may be 
of interest. 1,112 district!; were represented by the original Westat 
samples for the restandardization and equating studies combined. 
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(Some districts contained schools in both.) Twenty ineligible districts 
vere deleted, and nine supplemcntriry districta added, for an adjusted total 
of 1,101. A total of 1,096 districts actually accepted the invitation to 
participate (99.5%); of the number accepting, 1,023 (93,3%) were primary 
sample districts, and 73 (6.7%) were substitute districts. 

School r^^vita tions 

With the return of a district superinupndent's reply card to MS, 
gre^LHtions were begun for the maifing^f'"±nd^vi4u al school invi talion 
packets. School invitation packets, although addressed to individual 
principals, were mailed to the resp'Uisible district coordinator. This 
permitted direct distribution through the Jistrici: cffire, and school 
principals immediately became aware that the district had approved parci- 
cipat:.on. It also served to introduce the individual appointed as Disf .xct 
Coordinator. A cover letter, sent with these packets, requested thaf the 
district coordinator distribute the invitations to the schools, help in 
describing the project to the principals, and assure that the school (s) 
completed aod returned the reply forms contained in the invitation packet. 

The iiiVlcition letter requested ♦•liot: the school r-incipal approve parti- 
cT-pation for his/her school and appr^nt ar- individual to serve as School 
Coordinator. Thi? individual would have primary responsibility for the 
actual test administration and data collection. The School Coordinator 
would complete a Pre-Test Information iorm, be responsible for proper test 
administration, receive and distribute test materials, and complete the 
school and class summary forms; Since these tasks would involve other than 
class time, the School Coordinator would receive an honorarium basf:d on the 
number of pupils tested. 

School invitation packets were first released in December and mailings 
continued until April. Schools unable to accept the invitation were replaced 
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by schools dc:>ignated as back-up^s in the sam^-ie. Depending upon the charac- 
teristics o£ the originally selected school, the specific back-ups wei"e 
located either in the same district, or in auothe: district and/or state. 
In some cases, this required going back one step in the process and issuing 
an invitation to a district superintendent not previou&jy invited. 

D ii'lng die entire school invitation process, the district coordinators, 
state coordinators, and the USOE were kept informed of the status of all 
school invitations. 

In the Rettandardization Study, the original sample consisted of 940 
schoor^c. Of that n'^iabec. 27 were ineligible (closing, non-c::?.stent grades, 
etc.) and were not invitoc*., nor was any atteiipt made to obtain substitutes 
for them. , A special group of 178 distiicis provided a supplementary sample 
of 20 additional primary schools. Of the resulting group of ^33 schools, 
91P gr 98.4% actually tested and provid.-d usable data. Of these ?\S schools 
838 (91.3%) were originally selected (primary sample) schools, and 80 (8.7%) 
were substitute schools. Of the 202,756 pupils in these 918 schools, 95.1% 
(192,74?) ptcmided usable data. 

Fr.^ uiie conduct of tne equatirig Study, a total of 448 units, each 
contaiiing a '•^nimum of two classes at each of grades 4, ^, and 6 was 
required. When an individual school cild not fulfill this requirement, two 
or more schools were combined to form a "pseudo school", which then served 
as a sc'aooi-unit. Of the 448 units required, 445 (99- 3%) were obtained. 
The -445 units represented a total of 736 schools, c' wl«ich 639 (86.8%) were 
original (primary) sample schools and 97 (13.2%) were substitute schools. 
Of the 143,636 pupils in the 736 schools, 93.9% (134,855) provided usable 
data. 
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MAT EUT:^T, S PREPABAijxo^t y^^p SHIPPING 

School participat:*-»n Ar the Anchor Test Study was confirmtu by the return 
of an agreement postcard and a completed Pre-Test Information Form. The 
postCcird designated a School Coordinator, the contact for all subsequent Study 
transactions. The Pre-Te:;t Information Form served three purposes: it up- 
dated the school description f;.le on the USOE compr'er tapes, it punplip^ 
additiorai ir.put data required for the study, and it provided the inforn.c^tion 
needed to assign tests and prepar'^ ohxpments. 

■ fre-TesF Information Forms were received and recorded in a master control 
system, assuring complete returns from all participants. The forirs for 
Equating Study ''ere audited to assure the existence of two cla.sses each at 
grades 4, 5, and 6 and/or to assure that the grades represented were the 
appropriate caponent of a t-.i stin'^ unit (i.e., "pseudo-school")- All Pre-Test 
Information Forms were thoroughly edited, and school coordinators were con- 
tacted for any missing or incomplete information. All equating schools were 
randomly assigned to a particular test-pair, with the restriction that neither 
test assigned had already been administeieu as part of the school's o\n 
program. 

These test assignments, tcgether with th^ class descriptions, pL"yHde3 
the basi'* i-ironP.t tion necessary for the computer preparation of control 
forms and shipping orders. The shipping notice served as the master control 
form. Produced for each individual school, it listed the School Coordinator 
and Tesc Administrator kit? (materials) to be packaged and shipped. The 
School Coordinator was providt:*d vrLth a kit for each separate test form and 
level being administered at the school including a School Coordinator's 
Manual and Report Form, copies of the Test Administrator's Manuals, test books 
and answer sheets, envelopes for the return of answer sheets, and special 
a cknowledgement-of -recelp t cards . 
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The Test Ar^niLnistrator's Kit provided the materials necessary to admi-.iister 
one test to a class or group of students, including the appropriate Test 
Administrator's Manual, a Class Identification Form, and test books and answer 
sheets. 

A label v/as attached to each kit, identifying it as containing a specified 
test for a particular, teacher. In thp Norir^-lng Study, each teacher (clars or 
group test administrator) received one indi-n.dually addressed kit. 

oecause two tests were administered in the Equating Study, eac> teacucr 
received two personally addressed Icitsr ^Siu^:^ altetiiate cl asses ^ wTthin ar 
eauating school, were asked to administer tests in o.. alternate sequence, a 
second label was affixed to each kit in order to cici^ly identify which k'.t 
contained materials for the test tc be administered first and for th3 test 
to be cidm-..nistered second. 

In addition to the shipping notice, each school coordinator rec jived a 
computer-produced School Coordinator's Report Form. This form was designed 
to provide additional testing information and to help the coordinator organize 
the return c£ completed answer sheets LL listed all the class/ teacher 
titles and te^*-s, individually, and rti^^^ested that th^ coordinator use the 
LOim to verify complete retunib. r iordli4;'*"ois for norming schools were asked 
to indicate the ^rade level for each participating class group, the monber ;f 
students in the class actually tested, and the total number of pupils enrolled 
in the class. For participants in t!ie Kquatinj; Stu4y» the information was 
requested twice! once for each of the two tests administered. Equating Study 
participants were also asked to indicate the actual sequence of their test 
administrations, as well as the one test for which they preferred to have 
scores reported. A similar computer-produced Class Identification Form served 
as the cover document for each separate class group. 
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M/^l'VRT AL HECl'TPT/SCQRING AI^J) REPORTI NG 

The week of April 17th was designated as the week for schools to 
administer the Anchor Test Study instruments. The v;eeks of April 10th 
and April 24th were established as alternate administraf.ion weeks. Approxi- 
mately 85% of the participants were able to admi'^ir^ler ihe tt^sts during 
the pri'.rt *ry vej5ting (/eek. The alternate weeks wc.lc uc.ed by chose schools 
with program conflicts, and several school? in the Eastern area whose 

semester vacauions overlapped the primary administration week. Through 

immediate telephone follow-up, it was possible to obtain completed answer 
shee^f, and cover documenus> Trom 99.7% of those schools that hac agreed to 
parti cipate. 

The initial seep in material-return/ i.eceiving consistp'^ determining 
that answer sheets had been enclosed for ench participating class (one set 
of answei slice ts for each class in t\\^ Nor.ning Z'c idy ouJ iwo sees of answer 
sheets for each class in the Equating Study) and that no irregularities 
existed. The answer sheets were then forwarded for scanning and editing. 

Scoring and Reporting* 

Cpti^i'' transcriptiuii of header sheet and answer sheet information vas 
Lho •fp-''tial wrta processing step in the scoring and reporting cycle. Infor- 
mation was transcribed directly to magnttLc tape by an NCS optical transcrip- 
tion machine. Computer editing programs scanned the transcribed data and 
produce.' a ''.i.^ting of all student and/or school inform.icirn that appeared 
to be in conflict. This information was corrected, upaated, and re-introductd 
into the system by re-processing the original input documents. Editing 
specifications were implemented in order to assure "clean" and valid data. 
During the course of all processing, stringent quality control procedures 
were maintained in order to assure accuracy. 
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Wfiile the data for all students and schoolfj x^ere being processei^ and 
inc»tch^!d to the master cr-pAyvctr file for use in the subsequent analyses, 
basic score* reports vjere prjvidrd for immediate release to the participants. 
Based on each publishci 't- current normative data, a roster of results was 
prepared for each school. Schools participating in the Norming Study received 
rosters of individual Wo*-*^ Anal/sis, Reading, and loKtxl scores on the 
Metropolifin Pleading Test. Since each student in the Equating Stv.^y had 
taken two teots. the school was ailov?*^- ! to select the one test for which It 
Referred to havi scores reported. 

Score rosters (by class) provided raw scores, the publisher's national 
percentile rank, and a stanine score for each pupil tested, as ^ell as class 
summary da*, a 

In addition to the cla.«:s data for Pich school, a combined grade level 
summary was y^repared; This grade level re.port shov/cd the raw score mean, 
the standard deviation, y^j.Ll and the number and pc^j^CiiL otudencS 

falling in each quartile of the publisher's national norms (for all classes 
in that grade combined). In order to h^x^.. schools understand the score 
they were receiving, all reports included ^ description entitlpi' "Interpreting 
Ccore Reports". This document served to define the scores and iicirr^a^ive 
informatio-:, as w.ll as to explain the use of summary data. 

Roster preparation and release were scheduled to provide results to 
all schools as quickly as possible, hopefully prior to the end of thp school 
year. More than 90% of the school reports were released to participi^wto during 
the first week ixt June. The reports for tardy respondents and schools with 
irregularities were mailed during subsequent weeks. By June 19th, all school 
reports had been released and all school coordinators had been sent their 
honoraria for assisting the Anchor Test Study. Upon completion of score 
reporting, the data analyses for the Norming and Equating studies were begun. 



